Improved Limited Discrepancy Search

We present an improvement to Harvey and Ginsberg’s limited discrepancy search algorithm, which eliminates much of the redundancy in the original, by generating each path from the root to the maximum search depth only once. For a complete binary tree of depth d, this reduces the asymptotic complexity from O(y2”) to O(2”). Th e savings is much less in a partial tree search, or in a heavily pruned tree. The overhead of the improved algorithm on a complete b-ary tree is only a factor of b/(b 1) compared to depthfirst search. While this constant factor is greater on a heavily pruned tree, this improvement makes limited discrepancy search a viable alternative to depth-first search, whenever the entire tree may not be searched. Finally, we present both positive and negative empirical results on the utility of limited discrepancy search, for the problem of number partitioning. Limited Discrepancy Search The best-known search algorithms are breadth-first and depth-first search. Breadth-first search is rarely used in practice, since it requires space that is exponential in the search depth. Depth-first search uses only linear space, and thus is often used for exhaustive searches. For a very large tree, however, exhaustive search is not feasible. In that case, we would like to search as much of the tree as possible in the time available, and return the best solution found. Depth-first search is not necessarily the best choice in this case. For example, consider a binary tree with costs associated with the edges, where the cost of a node is the sum of the edge costs from the root to that node. We want to find a leaf node of lowest cost. The best known algorithm to solve this problem exactly is depth-first branch-and-bound, searching the lower-cost child of each node first. If the tree is too large to search completely, we would like to search those leaves that are most likely to be of lowest cost first. Assume that the edge costs are independent random variables uniformly distributed from zero to one. The expected value of the minimum of two such variables is l/3, and the expected value of the maximum is 2/3. Thus, if we order the children of a node, the expected value of the left branch is l/3, and the expected value of the right branch is 2/3. The leaf node with the lowest expected total cost is the leftmost. If d is the depth of the tree, its expected cost is d/3. The leaf nodes with the second-lowest expected cost are those with all left branches except for one right branch in their path from the root. The expected cost of these leaves is (d 1)/3 + 2/3. The second and third leaf nodes from the left are in this set, but so is the leaf reached by going right from the root, and then left thereafter. The nodes with the next lowest expected cost are those with exactly two right branches in their path from the root, with an expected cost of (d-2)/3+ 4/3. Thus, depth-first search does not search the leaf nodes in non-decreasing order of expected cost. Limited discrepancy search (LDS) (Harvey and Ginsberg 1995), does however. A discrepancy is a right branch in a heuristically ordered tree. The first path generated by LDS is the leftmost. Next, it generates those paths with at most one right branch from the root to the leaf. The next set of paths generated by LDS are those with at most two right branches, etc. This continues until every path has been generated, with the rightmost path being generated last. Figure 1 shows the sets of paths with 0, 1, 2, and 3 discrepancies in a binary tree of depth three. The first algorithm that searched a tree in this fashion, was called left-first search (Basin and Walsh 1993). Their formulation of the algorithm used exponential space, however, and hence is not practical for large searches. (Harvey and Ginsberg 1995) independently discovered a linear-space version of the algorithm, which they called limited discrepancy search, and recognized its more general applicability. Figure 2 gives a pseudo-code description of a single iteration of Harvey and Ginsberg’s original LDS algorithm (OLDS) on a binary tree. Its arguments are a node, and the number of discrepancies (k: for that iteration. This function is called once for each iteration, with I% ranging from zero to the maximum tree depth. LDS can be applied to any search problem where one branch from each node is preferred to its siblings. The simplest extension to a non-binary tree is to treat 286 Constraint Satisfaction From: AAAI-96 Proceedings. Copyright © 1996, AAAI (www.aaai.org). All rights reserved. ILDS (NODE, K, DEPTH) If NODE is a leaf, return If (DEPTH > K) ILDS (left-child(NODE), K, DEPTH-I) If (K ) 0) ILDS (right-child(NODE), K-l, DEPTH-I) Figure 1: Paths with 0, 1, 2, and 3 discrepancies Figure 3: Code for one iteration of improved LDS ems (NODE, K) If NODE is a leaf, return OLDS (left-child(NODE), K) If (K > 0) OLDS (right-child(NODE), K-l) Figure 2: Pseudo-code for one iteration of original LDS any branch except the leftmost as a discrepancy. Harvey and Ginsberg showed that if the left branch has a higher probability of containing a solution in its subtree than the right branch, then limited discrepancy search has a higher probability of finding a solution than depth-first search, for a given number of node generations. They also showed that it outperforms depthfirst search on a set of job-shop scheduling problems. Improved Limited Discrepancy Search The main drawback of the original formulation of LDS, OLDS, is that it generates some leaf nodes more than once. In particular, the iteration for Ic discrepancies generates all paths with Ic nf less right branches. Thus, each iteration regenerates all the paths of all previous iterations. For example, OLDS generates a total of 19 paths on a depth-three binary tree, only 8 of which are unique, as shown in Figure 2 of (Harvey and Ginsberg 1995). A s an extreme case, while the rightmost path is the only new path in the last iteration, OLDS regenerates the entire tree on this iteration. Given a maximum search depth, the algorithm can be modified so that each iteration generates only those paths with exact/y k discrepancies. This is done by keeping track of the remaining depth to be searched, and if it is less than or equal to the number of discrepancies, exploring only right branches below that node. The modified pseudo code is shown in Figure 3. The depth parameter is the depth to be searched below the current node. It is set to the maximum depth of the tree in each call on the root node. Every leaf node at the maximum depth is generated exactly once by this improved version of LDS, ILDS. Leaf nodes above the maximum depth, however, as well as interior nodes, are generated more than once. This interior node overhead is analyzed below. Analytic Comparison of the Algorithms Here we compare the performance of these two versions of LDS analytically, to see how much is saved by the improvement. To do this, we count the number of leaf nodes generated by the two algorithms on a complete binary tree of depth d. Since each iteration of ILDS generates those paths with exactly k discrepancies, each leaf node is generated exactly once, by the iteration corresponding to the number of right branches in its path from the root. Since a binary tree of depth d has 2d leaf nodes, ILDS generates 2d leaves. This is also the asymptotic time complexity of the algorithm, since the interior node generations don’t affect the asymptotic complexity, as we’ll see later. The number of leaves generated by OLDS is more complex. First, we need to count the number of paths with k discrepancies. There is one path with no discrepancies, the leftmost one. There are d paths with one discrepancy, since the single right branch could occur at any level in the tree. In general, there are (3 branches with k discrepancies, the number of ways of choosing k right branches out of d branches. To search a tree of depth d, d+ 1 iterations of OLDS are needed, since the number of discrepancies can range from 0 to d. The single path in the zeroth iteration is generated d + 1 times, once in every iteration. The d paths in the first iteration are each generated d times, once in each iteration except the first, etc. In general, let 1~: be the number of paths generated by OLDS in a complete search to depth d. Writing the same equation in reverse order, Since (I) = (ddk), adding the two equations gives 22 = (d + 2) (;) +@+2)(;) +-a+(d+$) Since the sum of the binomial coefficients is 2d, 22 = (d+2)2d or d+22d z = 2 This is the asymptotic time complexity of OLDS, since the interior node generations don’t affect the